> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/jaypopat/cf_ai_duet/llms.txt
> Use this file to discover all available pages before exploring further.

# AI assistant

> Integrated AI pair programmer powered by Cloudflare Workers AI

## Overview

Duet includes an AI assistant that acts as an additional pair programmer in your session. It's powered by Meta's Llama 3 8B model running on Cloudflare's edge network, providing fast responses without leaving your terminal.

## Activating the AI

Press `Ctrl+G` while in a room to open the AI input prompt:

<Steps>
  <Step title="Open AI input">
    Press `Ctrl+G` in the terminal. You'll see:

    ```
    Ask the AI... |
    ```
  </Step>

  <Step title="Type your question">
    Ask anything about your code, request help with commands, or get debugging suggestions:

    ```
    How do I list all files modified in the last hour?
    ```
  </Step>

  <Step title="Submit with Enter">
    The AI will respond in the sidebar. Press `Esc` to cancel without sending.
  </Step>
</Steps>

<Info>
  The AI assistant requires a Cloudflare Worker URL to be configured when starting the Duet server:

  ```bash theme={null}
  duet --worker https://duet-cf-worker.your-subdomain.workers.dev
  ```
</Info>

## AI architecture

The AI runs as a Cloudflare Durable Object, maintaining conversation state per room:

```typescript theme={null}
export class DuetAgent extends Agent<Env, DuetAgentState> {
  override initialState: DuetAgentState = { messages: [] };

  private async runAI(messages: AIMessage[]): Promise<string> {
    const result = await this.env.AI.run("@cf/meta/llama-3-8b-instruct", {
      messages,
    });
    return result.response?.trim() || "";
  }
}
```

<Tabs>
  <Tab title="Model">
    **Llama 3 8B Instruct**

    * 8 billion parameter model
    * Optimized for instruction following
    * Runs on Cloudflare's global network
    * Typical response time: 1-3 seconds
  </Tab>

  <Tab title="Context window">
    Duet keeps the last 10 messages in context:

    ```typescript theme={null}
    ...this.state.messages.slice(-10).map<AIMessage>((m) => ({
      role: m.role === "agent" ? "assistant" : "user",
      content: m.text,
    }))
    ```

    This balances memory usage with conversation coherence.
  </Tab>

  <Tab title="System prompt">
    ```typescript theme={null}
    {
      role: "system",
      content:
        "You are Duet, a concise pair-programming assistant. " +
        "You can run commands in a sandbox using <run>command</run> tags. " +
        "When asked to perform an action, briefly explain what you will do " +
        "and wrap the exact shell command(s) in <run> tags. " +
        "Do NOT include predicted output in your response - just provide " +
        "the explanation and command."
    }
    ```
  </Tab>
</Tabs>

## AI sidebar

The AI chat appears in a collapsible sidebar on the right side of your terminal:

```
┌─────────────────┬──────────────────────┬────────────────────┐
│   Users         │   Terminal           │   AI Assistant     │
│                 │                      │                    │
│   alice (host)  │   $ ls -la           │   You: How do I... │
│   bob           │   total 48           │                    │
│                 │   drwxr-xr-x ...     │   AI: You can use..│
└─────────────────┴──────────────────────┴────────────────────┘
```

### Keyboard shortcuts

| Key      | Action                       |
| -------- | ---------------------------- |
| `Ctrl+A` | Toggle AI sidebar visibility |
| `Ctrl+J` | Scroll chat down (3 lines)   |
| `Ctrl+K` | Scroll chat up (3 lines)     |
| `Ctrl+G` | Open AI input prompt         |

<Note>
  The AI sidebar only appears when your terminal is at least 120 columns wide and 24 rows tall. On smaller windows, the feature is automatically disabled.
</Note>

## Command execution

The AI can execute commands in a Cloudflare Sandbox by wrapping them in `<run>` tags:

```typescript theme={null}
private async executeCommands(text: string, roomId: string): Promise<string> {
  const matches = Array.from(text.matchAll(/<run>([\s\S]*?)<\/run>/g));
  let result = text;

  for (const match of matches) {
    const cmd = match[1]?.trim();
    if (!cmd) continue;

    try {
      const sandbox = getSandbox(this.env.Sandbox, `sandbox-${roomId}`);
      const { stderr, stdout } = await sandbox.exec(cmd);

      const summary = stdout.slice(0, 500) || stderr.slice(0, 500) || "[no output]";
      result += `\n\nOutput (${cmd}):\n${summary}`;
    } catch (e) {
      result += `\n\nError (${cmd}):\n${e.message}`;
    }
  }
  return result.replace(/<run>[\s\S]*?<\/run>/g, "").trim();
}
```

### Example conversation

<CodeGroup>
  ```text User theme={null}
  How do I find all Python files?
  ```

  ```text AI Response theme={null}
  I'll search for Python files in the current directory and subdirectories:

  <run>find . -name "*.py"</run>

  Output (find . -name "*.py"):
  ./src/main.py
  ./tests/test_api.py
  ./utils/helpers.py
  ```
</CodeGroup>

<Warning>
  Commands run in an isolated Cloudflare Sandbox, not your shared terminal workspace. The sandbox is ephemeral and destroyed when the room ends.
</Warning>

## Conversation persistence

AI messages are synchronized across all participants in real time:

```go theme={null}
case AIResponseMsg:
    if m.currentRoom != nil {
        m.currentRoom.SetAIMessages(msg.Messages)
        // Notify other clients to sync their viewport
        m.currentRoom.BroadcastEvent(room.RoomEvent{
            Type: "ai_sync",
        }, m.clientID)
    }
    m.syncAIViewportContent()
    m.scrollToLastPrompt()
```

When someone asks the AI a question:

1. The response is stored in the room's shared state
2. An `ai_sync` event is broadcast to all participants
3. Everyone's AI sidebar updates to show the new messages

<Info>
  Late joiners can see the full AI conversation history when they enter the room. The last 50 messages are kept in memory.
</Info>

## Message format

Each message in the conversation includes:

```go theme={null}
type AIMessage struct {
    Role   string `json:"role"`    // "user" or "agent"
    UserID string `json:"user_id"` // Username of the person who asked
    Text   string `json:"text"`    // Message content
    Ts     int64  `json:"ts"`      // Unix timestamp
}
```

This allows the UI to display who asked each question:

```
You: How do I list files?

AI: Use the ls command...

alice: What about sorting by date?

AI: Add the -t flag...
```

## API endpoints

The Cloudflare Worker exposes these AI endpoints:

<ParamField path="POST /api/rooms/:roomId/message" type="endpoint">
  Send a message to the AI

  **Request body:**

  ```json theme={null}
  {
    "text": "How do I list files?",
    "userId": "alice"
  }
  ```

  **Response:**

  ```json theme={null}
  {
    "reply": "Use the ls command to list files...",
    "messages": [
      {"role": "user", "userId": "alice", "text": "How do I...", "ts": 1234567890},
      {"role": "agent", "text": "Use the ls command...", "ts": 1234567891}
    ]
  }
  ```
</ParamField>

<ParamField path="DELETE /api/rooms/:roomId" type="endpoint">
  Clean up AI state and sandbox for a room

  Called automatically when the last participant leaves.

  **Response:**

  ```json theme={null}
  {
    "cleaned": true,
    "roomId": "a3f8e9d2-..."
  }
  ```
</ParamField>

## Client implementation

The Go client communicates with the Worker:

```go theme={null}
func (c *Client) SendMessage(ctx context.Context, roomID, text, userID string) (*MessageResponse, error) {
    url := fmt.Sprintf("%s/api/rooms/%s/message", c.baseURL, roomID)

    body := MessageRequest{
        Text:   text,
        UserID: userID,
    }

    jsonBody, err := json.Marshal(body)
    req, err := http.NewRequestWithContext(ctx, http.MethodPost, url, bytes.NewReader(jsonBody))
    req.Header.Set("Content-Type", "application/json")

    resp, err := c.http.Do(req)
    // ... handle response
}
```

Requests timeout after 30 seconds to prevent hanging if the Worker is slow.

## Error handling

<AccordionGroup>
  <Accordion title="No Worker URL configured">
    If you press `Ctrl+G` without a Worker URL:

    ```
    AI not configured (no worker URL)
    ```

    Start the server with the `--worker` flag.
  </Accordion>

  <Accordion title="Request timeout">
    If the AI doesn't respond within 30 seconds:

    ```
    Error: context deadline exceeded
    ```

    Try again or check your Worker status.
  </Accordion>

  <Accordion title="Validation error">
    Empty messages are rejected:

    ```json theme={null}
    {
      "error": "invalid request",
      "details": {"text": ["Text cannot be empty"]}
    }
    ```
  </Accordion>
</AccordionGroup>

## Best practices

<CardGroup cols={2}>
  <Card title="Be specific" icon="bullseye">
    Instead of "help with this code", ask "how do I parse JSON in Go?"
  </Card>

  <Card title="One question at a time" icon="one">
    The AI works best with focused questions. Break complex tasks into steps.
  </Card>

  <Card title="Include context" icon="book">
    Mention what you're trying to do: "I'm debugging a Python script that..."
  </Card>

  <Card title="Review commands" icon="check">
    Always verify AI-suggested commands before running them in your terminal.
  </Card>
</CardGroup>

## Limitations

* The AI doesn't have access to your terminal history or current directory
* It can't see files in your workspace (only in its own sandbox)
* It doesn't remember conversations across different rooms
* The 10-message context window means very long discussions may lose coherence

## Next steps

<CardGroup cols={2}>
  <Card title="Sandbox execution" icon="cube" href="/features/sandbox-execution">
    Learn how the AI's command execution sandbox works
  </Card>

  <Card title="Deploy a Worker" icon="cloud" href="/deployment/cloudflare-worker">
    Set up your own Cloudflare Worker for AI features
  </Card>
</CardGroup>